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FROM  AUTONOMOUS  NAVIGATION  TO  MULTI  PLATFORM  COOPERATION 

In  the  ReSSAC  project,  two  Yamaha  RmaX  remote  control  rotorcraft  have  been  equipped  with  an  autonomous  control 
architecture  and  algorithms  providing  capabilities  of  autonomous  flight,  navigation,  take-off  and  landing  in  known  areas. 
The  project  is  now  working  on  the  problem  of  autonomous  landing  in  an  unprepared  area,  which  requires  autonomous 
reactive  flight  and  navigation  capabilities  in  order  to  react  to  obstacles  on  the  one  hand,  and  some  autonomous  decision 
making  capabilities  in  order  to  cope  with  environment  uncertainties  on  the  other  hand.  The  aircraft  actions  may  have 
uncertain  results  due  to  noisy  measurements,  wind  or  turbulence,  but  also  because  the  terrain  is  ill-known,  potentially 
obstructed  by  obstacles.  The  ReSSAC  rotorcraft  are  using  flight  and  navigation  sensors  (GPS,  compass,  accelerometers 
and  gyrometers)  to  estimate  the  current  navigation  state,  but  it  can  also  perceive  the  surrounding  environment  thanks  to  an 
embedded  image  acquisition  and  processing  vision  system  (from  camera  to  decision  computer).  Additional  experiments 
have  been  conducted  in  order  to  simulate  in  flight  the  decking  of  a  rotorcraft  on  a  virtual  mobile  point,  the  motion  of 
which  is  emulated  and  measured  by  a  distant  moving  device  including  an  hybrided  GPS  and  Inertial  Measurement  Unit. 
Relative  localization  of  potential  obstacles  is  achieved  by  monocular  stereovision  at  higher  altitudes  and  optic  flow  based 
algorithms  at  lower  ground  heights.  The  ReSSAC  autonomous  rotorcraft  adapt  to  the  environment  according  to  a 
conditional  action  strategy,  obtained  as  the  optimal  solution  of  a  structured  Markov  Decision  Process  (MDP),  using  also 
the  initial  mission  definition  and  an  itinerary  graph  generated  from  the  a  priori  numerical  terrain  model.  The  aircraft 
control  and  supervision  architecture  is  justified  by  dependability  concerns.  Experiments  of  landing  in  the  vicinity  of 
artificial  obstacles  have  been  performed,  leading  to  improve  the  robustness  of  the  system,  using  a  progressive  exploration 
of  the  environment  in  three  phases,  including  different  vision  algorithms  and  online  mission  planning  which  have  been 
demonstrated  in  flight.  This  work  opens  the  ways  for  cooperative  decision  making  for  multi  platform  cooperation. 


1.  INTRODUCTION 

1.1  Autonomy:  what  for  ? 

Network  centric  operations  of  multiple  UAVs  and  platforms,  require  higher  levels  of  autonomy  (Figure  1)  in 
order  to  play  a  significant  role  in  future  warfare  operations,  homeland  security  or  risks  management 
applications  (see  references  [3]  to  [11]).  UAV  mission  planning  tools  have  been  developed  in  recent  years  in 
the  Systems  Control  &  Flight  Dynamics  Department  of  ONERA.  Specific  mission  planning  algorithms  have 
been  developed  and  integrated  in  the  ReSSAC  project's  mission  planning  decision  aid  software  of  the  ground 
control  station  (see  references  [14]  to  [19]).  However,  the  resulting  decision  aid  tools  do  not  really  solve  the 
safety  problems. 

The  management  of  safety  issues  via  ground  operated  fail  safe  procedures  is  prone  to  lead  to  undesired 
mission  interruptions.  Situation  awareness  problems  may  lead  the  security  operators  to  immediately  apply 
unnecessary  drastic  measures  (false  fault  detection,  temporary  loss  of  control  data  link,  ...).  More  flexibility 
and  time  for  analysis  would  be  required.  This  in  turn  requires  safe  and  efficient  capabilities  of  on-board  fault 
detection,  identification  and  autonomous  reconfiguration. 
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Figure  1  Autonomy  levels  hierarchy 


1.2  ReSSAC  Project 

The  ReSSAC  project  at  ONERA  ([1],  [2],  [13])  aims  at  demonstrating  the  feasibility  of  enabling  uninhabited 
systems  with  embedded  perception  and  decision  making  capabilities. 

Two  remote  control  Yamaha  RmaX  helicopters  (Figure  2)  have  been  bought  as  experimental  platforms. 


Figure  2  One  of  ONERA's  two  autonomous  RmaX  rotorcraft 
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Both  ReSSAC  uninhabited  helicopters  (one  of  which  is  shown  on  Figure  2)  are  equipped  with  onboard 
avionics  systems  providing  them  with  the  capabilities  of  autonomous  take-off,  autonomous  flight,  autonomous 
navigation  and  autonomous  landing  in  known  areas  and  known  environments.  Their  control  architecture, 
including  the  security  ground  pilot,  the  ground  station  and  data  links  have  been  developed  with  strong  safety 
and  security  requirements.  Such  capabilities  do  ensure  a  maximum  level  of  operational  and  functional 
autonomy  throughout  the  successive  flight  phases  of  most  possible  missions,  but  only  under  the  close 
supervision  of  operators.  In  such  a  context,  all  the  mission  reconfigurations  have  to  be  managed  by  human 
decision  makers.  Since  the  possible  conditional  alternatives  may  rapidly  become  unmanageable  in  case  of  an 
emergency,  their  number  must  remain  limited.  The  aircraft  flight  plan  is  bound  to  be  constrained  within  strict 
boundaries,  obtained  via  rigid  authorization  processes  favoring  simple  and  robust  safety  procedures. 


This  paper  presents  our  current  efforts  to  propose  a  generic  approach  toward  UAV  decisional  autonomy  and 
safety  within  the  ReSSAC  project  [13] [14]  and  is  organized  as  follows. 

We  first  recall  the  autonomous  flight,  guidance  and  navigation  capabilities  of  the  ReSSAC  uninhabited 
RmaX  rotorcraft. 

We  then  describe  the  experiments  conducted  with  one  of  these  rotorcraft,  experiments  that  allowed  us  to 
demonstrate  in  flight  a  series  of  simulated  decking  on  a  virtual  mobile  point.  This  demonstrate  the  ability 
of  our  rotorcraft  to  apply  a  reactive  strategy  conditional  to  its  dynamic  external  environment. 

This  reactive  strategy  can  be  designed  to  perform  other  missions:  we  present  the  waypoints  and  path 
planning  algorithms  and  tools  of  the  ReSSAC  project.  These  tools  allow  us  to  automatically  plan  an 
exploration  mission  in  a  partially  known  region. 

The  rotorcraft  then  needs  to  perceive  its  environment,  prior  to  deciding  and  reacting:  our  vision  algorithms 
and  landing  site  characterization  image  processing  experiments  are  described.  These  algorithms  are  then 
used  for  the  automatic  selection  of  a  landing  spot  :  autonomous  landing  on  a  partially  known  terrain 
obstructed  with  obstacles  was  demonstrated  in  flight. 

Finally,  we  describe  the  on-going  work  in  order  to  provide  the  rotorcraft  with  the  capability  of  mission  re¬ 
planning  :  the  UAV  should  be  able  to  automatically  generate  the  reactive  strategy  in  order  to  explore  its 
environment  in  successive  phases,  re -plan  its  exploration  strategy  and  act  conditionally  to  its  environment 
:  land  on  the  terrain  if  appropriate  and  go  elsewhere  if  appropriate. 


2.  AUTONOMOUS  FLIGHT  AND  NAVIGATION 

In  the  ReSSAC  project,  two  uninhabited  Yamaha  RmaX  helicopters  have  been  equipped  with  an  autonomous 
navigation  and  guidance  architecture:  this  core  flight,  guidance  and  navigation  architecture  is  authorized  to  fly 
autonomous  under  the  supervision  of  a  security  pilot.  This  architecture  is  presented  with  more  details  in  [14] 
and  we  only  recall  some  features  here. 

The  ReSSAC  rotorcraft  are  able  to  fulfil  any  mission  described  in  terms  of : 

a  flight  plan  associated  with  procedures  (phases  and  flight  states  transition  to  comply  to) 
a  reactive  action  strategy,  conditional  to  external  events  or  perception  . 
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2.1  Flight  plans 

Flight  plans  are  defined  by  ’’enriched  waypoints”  to  be  reached,  each  waypoint  being  defined  by  : 
its  3D  location  (x,y,z)  in  earth  reference  coordinates 
the  desired  velocity  vector  Vc  and 
the  desired  heading  of  the  aircraft  at  this  point. 

In  order  to  determine  the  trajectory  between  each  of  those  waypoints,  we  apply  the  Principle  of  the  Maximum, 
with  a  minimum  time  criterion  under  the  constraint  of  saturation  imposed  on  the  yaw  rate.  In  the  horizontal  2D 
plan,  this  problem  can  be  solved  by  geometric  considerations,  because  of  the  optimality  of  a  “bang-bang” 
control.  This  gives  trajectories  formed  successively  by  a  first  arc  of  circle,  a  straight  line  and  a  final  arc  of 
circle,  as  shown  in  Figure  7:  the  determination  in  real  time  (at  each  guidance  step)  of  the  flight  parameters 
allows  the  real  time  computation  of  the  control  input  in  terms  of  yaw  rate  to  be  given  to  the  above  control 
loops. 

2.2  Guidance,  obstacle  avoidance  and  navigation 

The  SNAKE  (’’Systeme  de  Navigation  Autorisant  une  Kyrielle  d’Evitements”)  navigation  system  adds  to  the 
above  guidance  law  the  capability  of  avoiding  forbidden  or  dangerous  (even  moving)  areas. 

This  function  must  be  compatible  with  the  guidance  algorithm  and  with  its  use  in  real  time.  For  that  purpose, 
forbidden  areas  or  obstacles,  are  considered  as  circles,  or  pairs  of  circles,  to  be  geometrically  treated  one  by 
one  by  the  iterative  navigation  algorithm. 
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Figure  3  Avoidance  of  an  obstacle  or  a  forbidden  area 

For  each  obstacle,  two  possible  avoidance  trajectories  are  considered,  respectively  defined  by  the  ad  and  ag 
angles  (respectively  right  hand  side  and  left  hand  side  obstacle  avoidance  heading  as  in  Figure  3).  These  two 
angles  could  soon  be  measured  in  real  time  by  an  obstacle  detection  sensor,  but  it  is  not  the  case  yet.  The  two 
angles  are  computed  from  the  map  and  compared  to  the  current  heading  control  order  ac  to  conclude  on  a 
possible  collision  and  define  a  new  route  with  minimal  turn  : 
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If  ad<  ac<  (ad+  ag)/2  then  ac  =  ad 
If  ag  >  ac>  (ad+  ag)/2  then  ac  =  ag 

Forbidden  areas  (Figure  4)  and  obstacles  are  treated  by  order  of  decreasing  distance  to  the  aircraft  (the  most 
faraway  obstacle  is  treated  first  and  so  on).  The  final  heading  control  order  ac  is  given  when  all  the  forbidden 
areas  and  obstacles  between  the  next  waypoint  and  the  current  position  of  the  aircraft  have  been  treated.  This 
warrants  a  real  time  behavior  and  a  local  optimality  of  the  area  avoidance  trajectory,  even  with  moving 
forbidden  areas  or  obstacles:  it  does  not  always  provide  a  globally  optimal  (shortest)  trajectory  to  the  next 
waypoint,  depending  on  the  avoidance  areas  configuration. 
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3.  PROCEDURES  AND  SAFETY 
3.1  Operations 


The  behavior  of  the  rotorcraft  in  its  different  operation  mode  is  bound  to  follow  a  strict  procedure  for 
transiting  between  the  different  possible  operation  modes  and  mission  phases:  this  is  encoded  by  the  flight 
state  automaton  described  in  Figure  5. 


Other  such  state  automata  describe  the  overall  operation  rules.  The  automaton  presented  in  Figure  6  presents 
the  fail  safe  procedures  of  the  rotorcraft. 
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The  2D  mission  in  Figure  7  can  either  be  generated  automatically  as  explained  below,  or  be  designed  by  the 
operator:  three  waypoints,  linked  in  two  ways  for  a  final  loop  realization,  and  two  forbidden  areas. 


The  mission  progress  is  monitored  through  the  state  evolution  in  a  state  graph,  according  to  the  operator 
requests  and  the  safety  reports.  A  fail-safe  flight  is  engaged  by  the  safety  mechanisms  in  case  of  a  system 
failure.  The  possible  flight  states  corresponding  to  the  "autonomous  flight"  are  organized  as  in  Figure  5: 

three  ground  phases:  "stop",  "maintenance"  and  "initialization",  which  correspond  to  the  states  of  the 
helicopter  when  it  is  on  the  ground.  The  initialization  phase  includes  all  the  automatic  check-list 
procedures  before  take  off. 

two  transition  phases:  "take  off’  and  "landing". 

five  flight  phases:  “route”  and  “return  to  base”  which  use  the  SNAKE  navigation  and  three  other  phases 
linked  with  hovering  flight  (  "position  hovering  flight",  "velocity  hovering  flight",  "stand  by  before 
landing"). 

In  the  flight  state  "fail-safe  flight",  the  safety  procedures,  imposed  by  the  use  of  autonomous  vehicles,  are 
described  by  Petri  nets,  such  as  for  example  in  Figure  6  for  the  case  of  a  transmission  failure.  The  autonomous 
flight  capabilities  of  the  helicopter  allow  an  easy  definition  and  execution  of  these  procedures. 

For  example,  it  is  able  to  return  to  a  safety  landing  zone  without  any  operator  intervention  nor  predefined 
trajectory,  provided  that  the  map  of  known  obstacles  is  up-to-date  and  sound:  some  verification  of  the  map  is 
useful. 

The  verification  of  the  map  is  made  at  mission  planning  time  and  should  be  redone  as  often  as  the  mission  is 
re -planned  :  SNAKE  is  not  an  optimal  path  planner  and  obstacles  must  not  overlap  in  the  map  it  uses. 

3.2  Automatic  flight  authorizations 

The  mission  is  always  conducted  in  full  compliance  with  the  safety  and  operation  procedures.  This  is  the  basis 
on  which  the  rotorcraft  receive  their  automatic  flight  authorizations  from  the  French  Civil  Aviation 
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Authorities. 

According  to  these  authorities,  the  first  step  toward  decisional  autonomy  in  the  ReSSAC  projects  was  made 
through  our  attempts  to  improve  our  fail  safe  procedures  in  case  of  a  loss  of  data  link.  The  aircraft  was  given 
the  capability  to  automatically  re -plan  its  trajectory  toward  a  predefined  safe  landing  area.  The  French  civil 
aviation  authorities  considered  it  as  a  major  mission  reconfiguration,  leading  us  to  further  work  in  the  direction 
of  improving  both  autonomy  and  safety  guarantees  for  the  behavior  of  the  ReSSAC  autonomous  aircraft. 


Current  flights  are  still  performed  with  a  Mline-of-sightM  security  pilot,  and  authorized  on  the  basis  of  the 
autonomous  navigation  architecture  (described  in  the  following).  The  project  also  developed  a  mobile  ground 
station  allowing  future  " out-of-sight”  autonomous  flights  with  a  ’’inside”  security  pilot  enabled  to  control  the 
flight  of  the  aircraft  from  inside  the  mobile  ground  station  thanks  to  a  number  of  instrument  flight  screens  and 
a  front  camera  view  (Figure  8). 

The  step  to  ” out-of-sight”  flight  authorizations  only  depends  on  the  choice  of  a  military  flight  zone,  or  on  the 
development  of  an  independent  fail  safe  device  enabling  the  security  operator  to  rapidly  land  the  aircraft  in  a 
bounded  perimeter  in  case  of  an  emergency.  Such  a  device  is  available  only  in  ”line-of-sight”  operation  for  the 
moment  and  we  don’t  really  need  more  for  our  current  work. 


4.  MISSION  AND  ACTION  STRATEGY 

4.1  Mission  data  necessary  for  the  action  strategy 

From  the  ground  station,  the  operator  can  re-define  the  mission  by  providing  the  necessary  data  to  be  taken 
into  account  by  the  autonomous  navigation  system  : 

Waypoints  :  location  coordinates  (x,y,z),  velocity  vectors,  aircraft  headings 

Forbidden  areas  :  position  of  circles  centres  (one  or  two  circles  if  they  are  combined),  radius 
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Base  :  ground  station  for  a  normal  landing 
Safety  base  :  area  for  a  fail  safe  landing 

Path  :  list  of  waypoint  numbers  to  reach  successively  with  possible  loops  on  the  waypoints  and  actions  to 
apply  on  these  waypoints,  conditionally  to  external  events  or  perception. 

This  allows  the  uninhabited  helicopter  to  coherently  perform  two  possibly  incompatible  tasks: 

comply  to  the  established  mission  and  safety  procedures 

adapt  to  its  environment,  thus  showing  some  autonomous  behaviors. 

A  first  example  of  such  conditional  action  strategy  is  of  course  the  SNAKE  navigation  and  obstacle  avoidance 
strategy. 


4.2  Execution  of  an  action  strategy: 

Flight  simulated  decking  on  a  virtual  point  moving  approximately  like  a  "ship  at  sea" 


We  implemented  another  reactive  action  (decking)  strategy  in  experiments  conducted  in  order  to  flight  test  a 
ship  decking  strategy  based  on  GPS-IMU  measurement  made  on  both  the  air  vehicle  and  the  ship.  SNAKE 
was  still  active  (it  is  thus  shown  compatible)  for  obstacle  avoidance. 
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Figure  9  ReSSAC  flight  experiments  and  demonstrations  images  and  trajectories  records  of 
simulated  ship  rotorcraft  decking  on  a  designated  virtual  mobile  point 


The  experiment  shown  in  Figure  9  is  a  simulated  ship  decking. 


The  rotorcraft  applies  a  decking  strategy,  which  is  a  reactive  decision  rule  using  information  from  both: 

1 .  the  GPS-IMU  measurements  and  estimation  of  the  ship  location  and  motion 

2.  the  GPS-IMU  measurements  and  estimation  of  the  air  vehicle  location  and  motion. 

SNAKE  is  used  in  order  to  avoid  collision  with  the  simulated  ship  :  the  experimental  platform  was  not  virtual. 

The  ship  motion  is  simulated  by  a  pointing  device  moving  along  an  axis  on  the  experimental  platform.  This 
device  is  actuated  along  the  axis  and  along  two  rotation  degrees  of  freedom.  It  is  distantly  pointing  a  virtual 
point  which  is  bound  to  move  approximately  like  a  ship  at  sea,  under  sea  conditions  theoretically  up  to  level  5. 

The  decking  strategy  performs: 
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an  analysis  of  the  current  motion  of  the  virtual  point,  on  the  basis  of  the  information  provided  by  the  GPS- 
IMU  measurement  unit  moving  on  the  experimental  platform  (not  by  the  pointing  device  motion  emulator) 


a  5  seconds  in  advance  prediction  of  the  next  best  decking  "window”,  which  is  a  period  of  calm  motion 
appropriate  for  decking  ; 

a  decision  to  go  for  decking  :  "appontage”  if  appropriate 

then  a  decision  to  abort  decking  :  "repli”  if  the  ship  observed  current  motion  (during  the  air  vehicle 
manoeuvre)  becomes  incompatible  with  decking  condition  (thus  in  contradiction  with  the  prediction, 
which  remains  only  a  prediction). 

What  is  interesting  here  for  us  is  the  ability  of  the  rotorcraft  to  act  dynamically  and  conditionally  to  external 
events. 

The  hardest  task  was  for  the  rotorcraft,  which  had  to  act  according  to  : 
the  virtual  ship 
the  virtual  wind 
the  decking  strategy 
the  true  safety  procedures 
the  actual  wind  ("vent  dAutan”  near  Toulouse) 

the  absence  of  true  decking  platform  (both  easier  and  harder  because  it  had  to  stop  its  descent). 

After  our  experimentation  campaigns,  we  performed  a  full  day  (16th  July  2006)  of  demonstrations,  taking  into 
account  various  possible  orientations  and  approach  trajectories  for  the  rotorcraft  relative  to  the  virtual  ship, 
under  different  relative  wind  hypothesis. 


5.  WAYPOINT  AND  ITINERARY  PLANNER 

5.1  Mission  planning  tools 

Our  flight  experiments  and  demonstrations  show  the  ability  of  the  ReSSAC  rotorcraft  to  act  and  adapt  to  its 
environment  using  reactive,  or  conditional,  decision  strategies.  We  now  present  general  tools  developed  in 
order  to  design  the  aircraft  mission. 
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Figure  10  Trajectory  optimization  and  itinerary  planning  based 
the  initial  terrain  map  and  rough  numerical  terrain  model 


Mission  planning  tools  were  developed  in  the  ReSSAC  project.  In  the  current  implementation,  this  software  is 
running  on  the  ground  station  computers,  as  shown  in  Figure  10  and  Figure  11,  but  we  may  very  soon  have 
sufficient  on-board  computation  capabilities  so  as  to  have  it  embedded. 

This  mission  planner  is  dedicated  to  provide  a  set  of  automatically  generated  possible  waypoints,  linked  to 
each  other  by  a  set  of  possible  itineraries  following  feasible  trajectories  between  the  waypoints. 


Figure  11  Automatic  waypoint  generation  (left)  and  global 
mission  planning  interface  (right). 


A  geometric  algorithm  is  first  applied  in  order  to  automatically  search  for  crests  and  valley  on  a  3D  numerical 
terrain  model.  The  crests  and  valleys  lines  are  populated  with  a  number  of  waypoints.  The  itinerary  planner 
then  generates  a  graph  of  time  (or  fuel)  optimal  itineraries  between  these  waypoints,  by  taking  into  account 
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the  flight  dynamics  and  the  fuel  consumption  model  of  the  aircraft. 

A  path  planner  applies  standard  Heuristic  Dynamic  Programming  (Moore -Dijkstra)  algorithms  in  order  to 
generates  a  graph  of  itineraries  combining  shortest  path  itineraries  and  low  altitude  itineraries. 

The  waypoint  and  itinerary  planner  is  connected  by  an  interface  with  the  SNAKE  obstacle  avoidance  strategy: 
they  communicate  by  exchanging  flight  plans,  as  described  previously  in  this  paper,  as  shown  in  Figure  12. 
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«  Flight  and  mission  plan  »  = 
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Figure  12  Autonomous  information  processing  and  decision  architecture 


5.2  Mission  planning  and  action  strategy 

However,  the  SNAKE  navigation  and  obstacle  avoidance  strategy  is  only  one  possible  decision  strategy  for 
safe  and  optimal  navigation  missions.  Our  optimal  itinerary  planner  needs  to  be  connected  to  a  more  global 
mission  (and  perception)  planner  in  order  to  address  more  complex  missions. 

The  goal  of  such  an  integration  is  : 

1.  to  enable  the  system  to  plan  its  mission  according  to  the  known  3D  map  of  the  environment  and  known 
obstacles  on  the  one  hand,  and 

2.  to  plan  for  information  acquisition  in  order  to  update  the  current  map  on  the  other  hand. 

Safe  exploration  and  landing  on  an  unprepared  and  ill-known  terrain,  potentially  obstructed  with  obstacles,  is  a 
motivating  challenge.  First  it  has  potential  applications  for  manned  rotorcraft.  Second,  it  requires  to  provide 
the  aircraft  with  safe  reconfiguration  and  decision  capabilities. 

This  work  is  part  of  a  cooperative  research  with  NAS  A/ Army  ARP  project  [5]  [6]. 
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6.  PERCEPTION  :  VISION  ALGORITHMS 

6.1  Perception 

Perception  is  obviously  a  crucial  issue  for  the  system  to  be  able  to  act  in  appropriate  interaction  with  its 
environment.  Tools  and  algorithms  for  the  perception  of  the  environment  and  the  updating  of  an  unknown  map 
of  the  exploration  area  have  been  developed  in  the  ReSSAC  project. 

The  goal  is  to  obtain  information  on  the  presence  or  absence  of  obstacle  on  the  terrain,  and  if  possible  update 
an  elevation  map. 

The  terrain  map  is  based  on  an  "a  priori”  numerical  terrain  model.  This  terrain  model  is  bound  to  be 
embedded,  which  means  that  we  are  using  a  very  coarse  model  today  :  triangulation  with  40  meters  long  edges 
on  average,  with  corresponding  elevation  errors  in  the  middle  of  the  triangles. 

In  any  cases,  we  suppose  that  the  a  priori  embedded  terrain  model  is  very  likely  to  be  correct  only  with  a 
coarse  granularity,  which  means  that  the  precision  in  terms  of  terrain  elevation  is  not  compatible  for 
autonomous  landing 

In  matter  of  vision  algorithms,  three  main  approaches  have  been  explored  and  combined: 
dense  stereovision  from  motion 
sparse  stereovision  from  motion 
texture  based  zoning. 

All  three  can  be  implemented  in  different  ways,  with  more  or  less  refinements  and  subsequently  more  or  less 
computation  cost. 
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6.2  Stereovision  from  motion 


Baseline  :  B 
Height :  H 

Triangulation  : 

precision  is  proportional  to  B/H 


Monocular  stereovision  from  motion  : 
baseline  is  as  large  as  desired  but  imprecise 


Binocular  stereovision  (two  cameras): 
B  is  wellknown 

but  fixed  (low  precision  at  great  height) 


B 

< - > 


Figure  13  Principles  of  stereovision  :  binocular  (2  cameras)  and  monocular  (1  moving  camera) 


The  basic  principle  of  " stereovision  from  motion”  is  that  the  distance  of  observed  features  in  the  images  is 
estimated  on  the  basis  of  the  way  they  move  from  an  image  to  the  next  one  in  the  video  sequence.  Pairs  of 
images  are  correlated,  in  order  to  match  a  point  in  the  first  image  with  the  corresponding  point  in  the  following 
image  :  the  distance  of  that  point  from  the  camera  is  obtained  by  triangulation  as  in  Figure  13. 

A  camera  is  mounted  on  the  ReSSAC  rotorcraft  in  ’’nadir”  orientation,  which  means  that  it  is  pointing 
downward,  vertically  when  the  rotorcraft’s  attitude  is  calm  and  horizontal.  In  such  a  configuration,  the 
estimated  distance,  if  obtained,  corresponds  to  the  altitude  of  the  camera  relative  to  the  obstacles  on  the 
ground. 

Depending  on  the  way  this  principle  is  applied  in  the  image  matrix,  the  method  should  be  called  dense  or 
sparse. 

Both  methods  need  some  hypotheses  to  be  made: 
the  scene  is  mostly  planar,  except  for  obstacles, 

the  camera  and  scene  models  can  be  approximated  by  a  homographic  or  quadratic  2D  model, 
the  speed  of  the  aircraft  and  gross  height  above  the  ground  can  be  roughly  known. 


6.2.1  Dense  stereovision  from  motion 

Dense  ’’stereovision  from  motion”  (Figure  14)  algorithms  roughly  work  as  follows: 

correlation  of  a  pair  of  successive  images  in  order  to  compute  a  field  of  pixel  speed  vector  (optic  flow)  or 
in  order  to  separate  the  main  scene  motion  from  residual  motion  of  ’’outliers”  that  do  not  move  as  the  main 
flow  (so  the  importance  of  the  ’’mostly  planar  scene”  hypothesis), 
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computation  of  a  point  scoring  according  to  the  norm  of  the  optic  flow  or  according  to  the  intensity  of  the 
gradient  in  the  images  themselves  and  in  the  displaced  frame  difference  between  images, 

computations  and  estimation  of  the  relative  location  of  supposed  obstacles, 

projection  of  a  dense  set  of  points  identified  as  obstacles  or  not  on  the  terrain  map:  the  result  is  a  dense 
map  without  ’’holes”,  but  longer  to  compute. 


6.2.2  Sparse  stereovision  from  motion 

Sparse  ’’stereovision  from  motion”  (Figure  15)  algorithms  roughly  work  as  follows: 

selection  of  interest  points  in  the  image  by  Harris  filtering  using  the  image  texture,  which  is  related  to 
local  variations  of  pixels’  intensity, 

tracking  of  these  interest  points  from  an  image  to  the  next, 
computations  and  estimation  of  the  relative  location  of  supposed  obstacles, 

projection  of  a  sparse  set  of  points  identified  as  obstacles  on  the  terrain  map:  the  result  is  only  a  sparse 
map,  i.e.  with  potential  ’’holes”,  so  the  importance  of  the  ’’mostly  planar  scene”  hypothesis. 
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6.3  Action  and  perception  coupling 

In  all  cases,  it  is  necessary  and  preferable  to  also  use  information  from  the  rotorcraft  navigation  estimation  and 
guidance  (action)  in  order  to: 

accelerate  the  matching  algorithms  that  track  the  pixels  from  one  image  to  the  next 
get  an  order  of  magnitude  of  the  aircraft  altitude  relative  to  the  ground 

adapt  the  algorithm  to  the  speed  of  the  rotorcraft,  which  determines  the  distance  flown  between  two 
images. 


7.  LANDING  SITE  CHARACTERIZATION  :  ELEVATION  MAP 

7.1  Dense  vs.  sparse 

Figure  14  shows  the  elevation  map  obtained  from  a  sequence  of  images  using  dense  techniques  (a  sample  is 
shown  on  the  left).  Optical  flow  techniques  are  complementary  applied  in  order  to  speed  up  the  process  and 
identify  "higher  zones”  more  rapidly.  The  image  and  the  elevation  information  is  very  nice,  but  takes  several 
seconds  to  obtain. 

Figure  15  shows  the  elevation  map  obtained  using  sparse  techniques,  which  means  that  the  number  of  tracked 
interest  points  is  limited  on  purpose,  in  order  to  speed  up  the  algorithm. 

7.2  Landing  site  characterization  experiments 

Figure  16  shows  the  combination  of  dense  stereovision  from  motion  and  texture  detection  techniques  that  we 
use  in  the  first  level  exploration  phase.  These  techniques  identify  the  main  obstacles  in  the  scene  and  generate 
adequate  sub-zones  to  be  further  explored  and  characterized.  Based  on  the  obstacles  and  terrain  texture,  the 
sub-zones  are  generated  and  assessed  with  respect  to  the  probability  of  finding  an  adequate  spot  for  landing 
within  it. 

These  sub-zones  are  then  explored  according  to  an  exploration  strategy  :  this  strategy  is  to  be  optimized  on  the 
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basis  of  the  generated  obstacles  and  sub-zones,  taking  into  account  uncertainties  in  the  environment  and  risks 
of  fuel  shortage  (see  paragraph  below). 


obstacles 


partially  free  zones 
(a  priori  probabilities  of 
« landability  »  before 
further  exploration) 


Figure  16  Obstacle  detection  and  zoning  using  stereovision  and  texture 


Figure  17  shows  the  combination  of  techniques  that  is  currently  used  in  real-time  in  the  final  landing  phase. 
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8.  DEMONSTRATIONS  OF  SAFE  AUTONOMOUS  LANDING  ON  A  PARTIALLY 
KNOWN  NON-COOPERATIVE  TERRAIN 

The  vision  algorithms  in  Figure  17  were  successfully  applied  for  landing  site  characterization  in  the  series  of 
flight  demonstrations  performed  in  autumn  2006  showing  the  capability  of  the  ReSSAC  rotorcraft  to 
autonomously  land  on  a  partially  known  terrain  obstructed  with  (cardboard)  obstacles,  in  the  conditions 
illustrated  in  Figure  18. 

These  flight  demonstrations  of  the  ReSSAC  aircraft  have  demonstrated  with  some  success,  some  capabilities 
of  autonomously  deciding  on  the  spot  where  to  land 

Indeed,  it  did  demonstrate  a  full  autonomous  decision  making  capability  at  one  precise  moment  of  one 
experimentation:  this  when  the  aircraft  decided  by  himself  that  the  search  zone  that  had  been  given  to  him  to 
explore  was  just  a  little  bit  too  narrow.  The  landing  spot  selection  algorithms  simply  spotted  a  very  nice 
perfectly  free  of  any  obstacle  landing  point  just  on  the  boarder  of  the  search  zone:  which  was  absolutely 
clever,  but  unexpected. 

As  a  matter  of  fact,  our  flight  demonstration  was  more  intended  to  show  the  helicopter  acting  exactly  as 
scheduled  and  landing  in  the  middle  of  the  cardboard  obstacles,  rather  than  acting  on  his  own  with  true 
autonomy. 

The  ReSSAC  helicopter  having  taken  too  much  autonomy  was  condemned  to  land  in  the  middle  of  the 
cardboard  obstacles  during  the  demonstrations  performed  within  the  hours  following  the  incident.  The  event 
was  not  even  filmed  nor  publicly  presented.  We  must  state  here  loud  and  clear,  for  the  sake  of  our  automatic 
flight  authorizations,  that  this  act  of  independence  and  true  autonomy  was  still  under  the  tight  supervision  and 
control  of  our  security  pilot  (the  autonomous  rotorcraft  was  arrested  right  after). 
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And  what  next  ?  We  first  planned  to  extend  the  area  covered  with  cardboard  obstacles,  so  to  suppress  the  nice 
’’boarder  line”  landing  spots. 


9.  EXPLORATION  IN  A  PARTIALLY  KNOWN  UNCERTAIN  WORLD 

We  then  decided  to  perform  the  full  exploration  mission  in  several  phases,  including  the  re -planning  of  the 
mission  in  the  middle,  as  described  in  the  following  paragraph. 

9.1  Phase  One: 

initial  navigation  and  exploration  of  the  search  zone:  planned  using  our  waypoints  and  itinerary  planner 
generating  a  navigation  map  and  systematic  exploration  (sweeping)  trajectories  in  order  to  explore  the 
initial  search  zone  at  higher  altitude  (50  meters  in  fact) 

geometric  decomposition  of  the  search  zone  into  obstacles  and  sub-zones  of  interest,  further  assessed  in 
terms  of  probability  of  ’’goodness”  for  landing  purposes 

Mission  re -planning  by  optimizing  a  conditional  strategy  (reactive  decision  rule  under  uncertainty)  for  the 
exploration  of  the  thus  ranked  sub-zones,  taking  into  account  the  probabilities  of  failure  or  success  in 
finding  an  site  appropriate  for  landing,  together  with  other  probabilities  coming  from  the  changing  and 
uncertain  environment. 

9.2  Phase  Two: 

Execution  of  the  exploration  strategy,  choice  of  a  sub-zone  to  characterize  and  test  for  slope,  until  safe 
landing  in  the  search  zone,  or  safety  landing  in  the  safety  zone,  or  safe  return  to  base,  . . . 
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9.3  Phase  Three: 

Test,  by  use  of  a  laser  telemeter,  of  the  slope  of  the  sub-zone  eventually  selected  in  Phase  Two,  then  if  the 
slope  is  ok  proceed  with  selection  of  a  landing  spot  based  on  the  techniques  presented  in  Figure  17  and 
landing,  otherwise  resume  exploration  in  Phase  Two. 


This  approach  clearly  requires  the  system  to  be  able  to  replan  its  mission  according  to  the  encountered 
situation  and  according  to  environment  uncertainties. 

The  techniques  applied  for  that  purpose  are  described  in  the  following  :  the  algorithms  are  ready  and  efficient. 
The  exploration  planning  problem  is  modeled  in  a  formal  way,  using  the  Planning  Problem  Description 
Language  PDDL  2.1  including  probabilistic  action  effects  description  . 

These  algorithms  have  been  implemented  on  the  on-board  computer  and  integrated  in  the  ReSSAC  decision 
and  supervision  architecture  described  in  Figure  12  . 

Flight  demonstrations  have  been  conducted  between  February  and  March  2007  and  should  be  more  officially 
presented  by  the  beginning  of  July  2007. 

The  experiments  shown  in  Figure  18  correspond  to  the  last  phase  of  the  present  ReSSAC  rotorcraft 
autonomous  exploration  mission. 


10.  DECISIONAL  AUTONOMY 

Extending  the  autonomy  capabilities  of  an  unmanned  aircraft  is  not  a  goal  in  itself.  Such  efforts  are  driven  by 
the  need  for  the  aircraft  to  be  able  to  manage  by  itself  a  situation  were  the  control  and  supervision  link  is  lost. 

Preparing  for  this  eventuality  often  appears  as  crucial  when  considering  complex  missions  involving  a  mix  of 
manned  and  unmanned  assets,  and  especially  the  insertion  of  unmanned  aircraft  within  an  airspace  populated 
with  other  aircraft.  The  loss  of  control  can  be  due  to  a  data-link  failure  with  the  ground  control  station,  due  to 
an  excessive  workload  of  the  operators  or  due  to  the  fact  that  the  link  with  the  information  and  decision 
network  that  normally  supervises  the  mission,  does  not  provide  a  sufficient  level  of  situation  awareness  to  the 
operators,  or  does  not  allow  them  to  react  in  time. 

For  these  reasons,  we  study  the  case  were  the  pre-planned  mission,  should  be  reconfigured  on-board  the 
autonomous  aircraft  by  its  own  re -planning  capabilities. 

10.1  Re-planning 

10.1.1  Reactive  replanning 

The  basic  level  of  such  a  replanning  capability  is  well  represented  with  the  case  of  obstacle  avoidance. 
Assuming  that  we  are  able  to  detect  autonomously  the  obstacle,  or  that  we  are  informed  by  other  means  of  the 
presence  of  this  obstacles,  we  are  able  to  adapt  to  the  new  situation  with  the  SNAKE  navigation  system,  unless 
the  obstacles  come  into  tricky  configurations  so  that  some  obstacles  overlap,  which  is  possible  with  moving 
objects. 
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SNAKE  is  therefore  more  considered  as  a  general  navigation  and  obstacle  avoidance  reactive  strategy,  rather 
than  a  true  planning  tool.  Higher  level  motion  planning  capabilities  are  thus  required  on  top  of  it. 

State  of  the  art  algorithms  can  provide  good  real  time  motion  planning  solutions  to  that  problem  in  the  case  of 
one  single  aircraft  (multiple  aircraft  flight  planning  and  collision  avoidance  can  be  much  more  complex).  Yet, 
most  missions  are  not  purely  geometrical  trajectory  planning  problems  and  combine  other  variables,  some  of 
them  controllable  and  some  of  them  random  or  unpredictable. 


10.1.2  Deliberative  re-planning  :  re-optimizing  the  action  strategy 

Figure  12  shows  a  classical  decisional  architecture  for  autonomous  agents,  which  is  applied  to  the  ReSSAC 
aircraft.  It  is  important  to  notice  that  all  the  situation  assessment,  decision  making  and  image  processing 
functions  need  not  be  fully  embedded  on-board  the  aircraft,  nor  in  the  ground  control  station. 

There  is  likely  to  be  a  sharing  between  the  functions  that  are  required  to  be  on-board  the  aircraft  and  those  that 
are  required  to  be  kept  under  control  of  the  operator:  this  possibly  uneasy  choice  is  partly  studied  in  the 
ReSSAC  project. 


11.  AUTONOMOUS  EXPLORATION  PLANNING  PROBLEM 

11.1  Planning  problem 

In  order  to  get  a  first  idea  of  the  possible  benefits  of  using  Artificial  Intelligence  planning  techniques  for  real- 
world  autonomous  aircraft,  the  case  of  an  exploration  mission  is  studied  within  the  ReSSAC  project.  This 
example  is  abstracted  from  the  initial  Search  &  Rescue  application  scenario  of  the  project. 

An  exploration  mission  is  composed  of  a  problem  of  navigation  and  a  problem  of  information  acquisition  in  a 
partially  known  environment.  It  can  be  addressed  at  different  levels  of  modeling.  Motion  planning  and 
information  processing  aspects  are  important  when  a  sufficient  flow  of  sensory  information  is  available 
through  the  use  of  range  or  object  detectors,  3D  sensors,  ...  etc.  Our  focus  is  on  a  more  abstract  level:  on 
planning  under  uncertainty  for  both  motion  actions  and  information  acquisition  tasks,  which  is  a  crucial  issue 
for  autonomous  systems. 

11.2  Exploration  planning  problem 

In  our  exploration  planning  problem,  different  regions  are  identified,  such  as  in  Figure  19,  after  a  preliminary 
(higher  altitude)  exploration,  the  partition  of  the  initial  search  zone  into  sub-zones  and  the  ranking  of  these 
zones  in  terms  of  probability  of  ’’goodness”  for  landing  purposes. 

These  regions  require  to  be  exhaustively  or  partially  explored,  mapped  or  searched  for  the  presence  of  persons 
or  objects,  before  continuing  the  mission.  Information  acquisition  may  be  part  of  intermediate  goals  of  the 
mission,  and  may  also  impact  on  the  subsequent  tasks  or  navigation  actions. 
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Such  missions  can  be  modeled  at  a  higher  level  of  abstraction,  as  a  sequence  of  mission  phases,  tasks,  or 
macro-actions:  for  each  mission  phase,  the  system  needs  to  achieve  navigation  and  information  acquisition 
goals,  before  proceeding  with  one  of  the  possibly  following  phases. 

The  order  in  which  the  intermediate  sub-goals  or  tasks  must  be  achieved  can  be  free  or  constrained.  Thus,  they 
can  be  seen  as  pre-conditions  of  other  tasks:  this  implies  that  searching  for  a  good  mission  plan  (not  even  the 
optimal  one)  requires  to  explore  the  combinatory  number  of  all  possible  branches  in  an  AND-OR  graph  of 
possible  states  looking  like  a  conditional  version  of  the  flight  plan  in  Figure  7,  with  additional  possible 
’’information  acquisition”  flight  phases  in  the  middle. 


Furthermore,  each  intermediate  goal  must  be  achieved  while  minimizing  risks  and  costs  and  thus  by 
optimizing  the  navigation  and  the  action  strategy  while  taking  into  account  uncertainties  on  the  values  of  a 
number  of  random  or  unpredictable  variables. 

11.3  Formal  Markov  Decision  Process  modeling 

Our  exploration  planning  problem  contains  a  number  of  possible  tasks,  associated  with  rewards,  each  of  which 
can  be  obtained  once  in  turn,  no  matter  the  order,  before  reaching  the  goals,  and  final  rewards.  The 
achievement  of  these  tasks  is  not  certain  and  there  are  uncertainties  in  the  results  of  the  performed  actions. 

To  model  this,  we  need  to  introduce  binary  state  variables  depending  on  whether  the  goals  have  already  been 
achieved  or  not,  and  some  representation  of  the  uncertainties.  The  formalism  of  Markov  Decision  Processes 
(MDP)  can  be  applied  in  that  case,  but  in  a  factored  state  and  action  representation  based  on  state  and  action 
variables. 

In  our  exploration  planning  problem,  the  Markov  Decision  Process  (MDP)  model  is  based  on  the  following 
variables: 

1 .  Action  variables  : 

At:  rotorcraft  action  choice  at  time  step  t 
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2.  State  variables 

Et:  rotorcraft  estimated  fuel  level  at  time  step  t 

Rt:  the  region  being  explored  by  the  rotorcraft  at  time  t 

Pt:  the  current  best  landing  point  at  time  t 

Vt :  the  current  flight  velocity  of  the  rotorcraft  at  time  t 

11.4  Dynamic  Bayesian  Networks 

Planning  problems  that  are  modeled  with  such  an  hybrid  structure  combining  a  navigation  state  space  and 
orthogonal  mission  of  internal  state  variables  have  been  studied  in  [16]. 

Such  problems  of  sequential  decision  under  uncertainty  can  be  modeled  using  Dynamics  Bayesian  Networks 
(DBNs)  such  as  in  Figure  19,  or  in  many  problems  of  the  ICAPS?2004  probabilistic  track  planning  competition 
in  which  we  participated  with  preliminary  versions  of  our  algorithms  [17]. 

In  Figure  19,  the  arrows  arriving  to  Et+1  mean  that  the  probability  distribution  on  the  possible  values  of  E 
(energy  level  of  the  aircraft)  at  time  t+1  is  given  as  a  conditional  probability  depending  on  Vt,  Et,  Pt  and  the 
choice  of  the  action  At  chosen  at  time  t. 

Our  exploration  planning  problem  is  thus  modeled  in  a  factored  form  [16-18],  and  difficult  to  represent 
graphically,  except  for  the  interface  defined  in  Figure  19.  The  underlying  navigation  problem  is  still  present, 
but  the  dimension  of  the  state  space  is  increased.  The  navigation  goals  to  be  achieved  depend  on  the  values  of 
the  other  variables.  The  level  of  energy  autonomy  is  a  good  example  of  an  additional  state  variable  with  a  high 
impact  on  the  navigation. 

Flight  is  thus  limited  by  energy  autonomy,  and  the  aircraft  can  decide  to  return  to  its  base,  either  if  all  its  goals 
have  been  achieved,  or  if  it  is  running  short  of  fuel,  in  which  case  it  may  as  well  go  to  its  failsafe  or  emergency 
crash  base.  Other  random  variables  may  also  impact  the  navigation  decisions,  such  as  the  ground  height,  the 
presence  of  objects,  the  distance  to  obstacles  (especially  in  unknown  environments),  the  local  winds  and 
turbulence  conditions  which  may  allow  an  approach  to  landing  or  not,  etc. 

11.5  Optimization  algorithms 

Yet,  such  factored  Markov  Decision  Process  (MDP)  formulations  of  the  problem  may  lead  to  optimization 
problems  of  untractable  complexity. 

We  tested  many  algorithms  [17-18]  and  developed  a  symbolic  focused  dynamic  programming  (SFDP) 
algorithmic  scheme  (Figure  20  and  Figure  21),  which  enables  us  to  efficiently  find  a  feasible  solution  for  very 
large  factored  MDP  [19],  that  optimal  algorithms  cannot  tackle. 
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Figure  20  The  SFDP  algorithm  can  tackle 
the  problems  of  size  beyond  N°7  (from 
left  to  right) 
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Figure  21  The  SFDP  algorithm  can 
incrementally  improve  the  reactive  action 
strategy  expected  value 


We  developed  a  incrementally  de-focused  approach  that  first  finds  a  feasible  solution  to  the  problem  and  then 
incrementally  imposes  increasing  constraints  on  the  optimality  of  the  solution,  until  optimality  [20].  Further 
work  will  be  devoted  to  the  development  of  optimization  algorithms  to  tackle  hybrid  probabilistic  planning 
problems  including  continuous  state  variables,  time  and  concurrent  actions,  with  extension  to  cooperative 
multi-agent  planning. 

11.6  Online  replanning 

The  on-line  "almost  anytime”  re -planning  algorithm  has  been  integrated  on-board  within  the  overall  mission 
management  architecture  with  the  motion  planning  algorithms  and  navigation  system.  These  techniques  have 
been  tested  on  the  ground  first,  then  experimented  in  flight  including  exploration,  landing  zone  selection  and 
mapping,  autonomous  landing  in  an  unprepared  area  and  flight  back  to  the  base.  The  official  final  demo  is 
scheduled  by  the  beginning  of  July  2007,  which  is  still  a  challenge,  w.r.t.  the  supervision  of  "autonomy”. 


12.  CONCLUSION 

We  have  presented  the  current  status  and  achievements  of  the  Autonomous  Air  Vehicle  ReSSAC  project  at 
ONERA.  The  project  has  now  reached  a  maturity  stage,  with  two  aircraft  capable  of  autonomous  navigation 
under  the  supervision  of  a  security  operator. 

A  large  variety  of  experiments  were  conducted  and  real  steps  forward  achieved  within  the  last  month  which 
have  lead  to  flight  demonstrations  of  autonomous  exploration  and  landing  in  an  ill-known  unprepared 
environment,  with  several  exploration  phases  from  coarse  obstacle  detection  to  finer  site  characterization, 
including  the  zoning  of  the  search  area  into  sub-zone  and  the  online  mission  re -planning  to  adapt  exploration 
to  the  encountered  environment. 

By  this  demonstration,  the  ReSSAC  project  achieves  a  true  feasibility  step  of  autonomy  capabilities  for 
uninhabited  aircraft,  either  for  Search  &  Rescue  or  for  other  missions.  This  demonstration  now  makes  it 
possible  to  make  further  steps  towards  autonomous  agents  in  cooperation,  planning  and  deciding  for 
cooperative  action  in  the  achievement  of  a  common  mission. 
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Of  course,  this  also  leads  to  work  on  motion  and  action  coordination  functions,  which  are  necessary  to  operate 
multiple  platform  concurrently.  Decisional  autonomy  is  not  required  in  order  to  perform  formation  flying  or 
visual  servoing.  However,  some  adequate  capacities  of  collective  adaptation  and  cooperative  decision  making 
are  required  in  order  to  achieve  a  common  mission  requiring  the  resources  from  several  agents,  some  of  them 
uninhabited,  and  in  order  to  achieve  cooperative  work  by  these  agents  while  facing  unexpected  events, 
intrusions  by  non-cooperative  agents,  pop-up  threats  or  moving  dangers,  for  instance. 

Further  work  will  be  focused  in  the  following  directions: 

embedded  decision  and  replanning 

improved  mobility  in  non-cooperative  environments 

cooperative  embedded  coordination  and  decision  making 

dependability  and  safety  :  how  to  prove  the  safety  of  highly  reconfigurable  control  architectures. 
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